Applying Pattern-based Classification to Sequences of Gestures

The pattern-based sequence classification system (PBSC) identifies regularly occurring patterns in collections of sequences and uses these patterns to predict meta-information. This automated system has been proven useful in identifying patterns in written language and musical notations. To illustrate the wide applicability of this approach, we classify symbolic representations of speech-accompanying gestures produced by adults in order to predict their level of empathy. Previous research that focused on isolated gestures has shown that the frequency and salience with which individuals produce certain speech-accompanying gestures are related to empathy. The current research extends these analyses of single gestures by investigating the relationship between the frequency of multi-gesture sequences of speech-accompanying gestures and empathy. The results show that patterns found in multi-gesture sequences prove to be more useful for predicting empathy levels in adults than patterns found in single gestures. This paper thus demonstrates that sequences of gestures contain additional information compared to gestures in isolation, suggesting that empathic people structure their gestural sequences differently than less empathic people. More importantly, this study introduces PBSC as an innovative, effective method to incorporate time as an extra dimension in gestural communication, which can be extended to a wide range of sequential modalities.